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Abstract 

In this paper we prove large and moderate deviations principles for the recursive kernel estimators 
of a probability density function defined by the stochastic approximation algorithm introduced by 
| Mokkadem et al. [2009. The stochastic approximation method for the estimation of a probability 

density. J. Statist. Plann. Inference 139, 2459-2478]. We show that the estimator constructed using 
f^*) ' the stepsize which minimize the variance of the class of the recursive estimators defined in Mokkadem 

, et al. (2009) gives the same pointwise LDP and MDP as the Rosenblatt kernel estimator. We provide 

(— i ■ results both for the pointwise and the uniform deviations. 
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H 1 1 Introduction 

c/3 . 

Let Xi,. . . ,X n be independent, identically distributed R d - valued random vectors, and let / denote the 
probability density of X\. To construct a stochastic algorithm, which approximates the function / at 
a given point x, Mokkadem et al. (2009) defined an algorithm of search of the zero of the function 
h : y i-> f(x) — y. They proceed as follows: (i) they set /o(x) G K; (ii) for all n > 1, they set 



•3 



fn(x) = /n-l(x) + 7„W„(x) 

\ where W n (x) is an "observation" of the function h at the point /„_i(x) and (7 n ) is a sequence of positive 

■ real numbers that goes to zero. To define W n (x), they follow the approach of Revesz (1973, 1977) and 
C*~) \ of Tsybakov (1990), and introduced a kernel K (which is a function satisfying J Rd K(x)dx = 1) and a 

■ bandwidth (h n ) (which is a sequence of positive real numbers that goes to zero), and they set W n (x) = 
h^K^k^lx — X n \) — f n -i(x). The stochastic approximation algorithm introduced in Mokkadem et al. 

■ (2009) which estimate recursively the density / at the point x is 



_ d „ / X-Xr, 



m i 

/„(x) = (1 - 7n)/„-i(x) + ln K d K (j-f^ ) ■ (1) 
• I— I ' 

Recently, large and moderate deviations results have been proved for the well-known nonrecursive kernel 
density estimator introduced by Rosenblatt (1956) (see also Parzen, 1962). The large deviations principle 
has been studied by Louani (1998) and Worms (2001). Gao (2003) and Mokkadem et al. (2005) extend 
these results and provide moderate deviations principles. The purpose of this paper is to establish 
large and moderate deviations principles for the recursive density estimator defined by the stochastic 
approximation algorithm ([T]). 

Let us first recall that a M m -valued sequence {Z n ) n>1 satisfies a large deviations principle (LDP) 
with speed (v n ) and good rate function / if : 

1. (y n ) is a positive sequence such that lim„_ ) . 00 v n = oo; 

2. / : R" 1 — > [0, oo] has compact level sets; 

3. for every borel set B C M. m , 

- inf I (x) < lim inf v~ x log P [Z n e B] 

o n— >oo 

< lim sup v~ x log P [Z n G B] < - in{_I (x) , 

n— >oo x£B 
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where B and B denote the interior and the closure of B respectively. Moreover, let (v n ) be a 
nonrandom sequence that goes to infinity; if (v n Z n ) satisfies a LDP, then (Z n ) is said to satisfy a 
moderate deviations principle (MDP). 

The first aim of this paper is to establish pointwise LDP for the recursive kernel density estimators 
defined by the stochastic approximation algorithm ([T]). It turns out that the rate function depend on 
the choice of the stepsize (7^); In the first part of this paper we focus on the following two special cases 

: (1) (7«) = { n ~ 1 ) an d (2) (j n ) = (h-t (J2k=i ^k) J> tne nrst OIie belongs to the subclass of recursive 
kernel estimators which have a minimum MSE or MISE and the second choice belongs to the subclass 
of recursive kernel estimators which have a minimum variance (see Mokkadem et al., 2009). 

We show that using the stepsize (j n ) = (n" 1 ) and (h n ) = (cn~ a ) with c > and a € ]0, l/d[, the 
sequence (/„ (x) — f (x)) satisfies a LDP with speed (nh^) and the rate function defined as follows: 

[if /(x)^0, I a>x :t^f(x)I a (l+jfa) 

\ if f (x) = 0, I a , x (0) = and I a , x (t) = +00 for t ? 0. 

where 

I a (t) = sup {ut - ip a (u)} 

^ a (u) = I s- ad ( e usadK W - 1) dsdz, 

which is the same rate function for the LDP of the Wolverton and Wagner (1969) kernel estimator (see 
Mokkadem et al., 2006). 

Moreover, we show that using the stepsize (7„) = (h^ (J2k=i ^t) ^ an< ^ more general bandwiths 
defined as h n = h (n) for all n, where h is a regularly varing function with exponent (—a), a G ]0, l/d[. 
We prove that the sequence (/„ (x) — f (x)) satisfies a LDP with speed {nhfy and the rate function 
defined as follows: 

(if /(x)^0, I x : t ->f(x)l(l + jfa) (3) 
X if f{x) = 0, J x (0) = and I x (t) = +co for t^0. 



where 



/ (t) = sup {ut — V> (u)} 

.had \ / 



which is the same rate function for the LDP of the Rosenblatt kernel estimator (see Mokkadem et al., 
2005). 

Our second aim is to provide pointwise MDP for the density estimator defined by the stochastic 
approximation algorithm |T]). In this case, we consider more general stepsizes defined as j n — 7 (n) for 
all n, where 7 is a regularly function with exponent (—a), a £ ]l/2, 1]. Throughout this paper we will 
use the following notation: 

£ = lim (n7„) _1 . (4) 

n— >-+oo 

For any positive sequence (v„ ) satisfying 

lim v n = 00 and lim " - = 

n— >oo n— ¥00 h 

n 

and general bandwidths (h n ), we prove that the sequence 

V n (fn(x) -/(«)) 
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satisfies a LDP of speed (h^/ {inV^)) and rate function J a , a ,x (■) defined by 

f «/ f (x) ^ 0, J a ,a,x ■ t 2 f(x)fl d K^lfdz ( 5 ) 

{ if f (x) = 0, J a ,a,x (0) = and J a , a , x (t) = +oo for t ^ 0. 



Let us point out that using the stepsize (7„) = I /i^ ^J_fc=i hf^J ^ which minimize the variance of /„, 

we obtain the same rate function for the pointwise LDP and MDP as the one obtained for the Rosenblatt 
kernel estimator. 

Finally, we give a uniform version of the previous results. More precisely, let U be a subset of M. d ; we 
establish large and moderate deviations principles for the sequence (sup^,-^ |/„ (x) — f {x)\). 

2 Assumptions and main results 

We define the following class of regularly varying sequences. 

Definition 1. Let 7 G R and (v n ) n>1 be a nonrandom positive sequence. We say that (v n ) G QS (7) if 



lim n 

n— >+oo 



V n -1 



7. (6) 



Condition ^ was introduced by Galambos and Seneta (1973) to define regularly varying sequences 
(see also Bojanic and Seneta, 1973), and by Mokkadem and Pelletier (2007) in the context of stochastic 
approximation algorithms. Typical sequences in QS (7) are, for b G K, n 1 (logn) , n 1 (log log n) , and so 
on. 



2.1 Pointwise LDP for the density estimator defined by the stochastic ap- 
proximation algorithm ([1]) 

2.1.1 Choices of (j n ) minimizing the MISE of f n 

It was shown in Mokkadem et al. (2009) that to minimize the MISE of f n , the stepsize (7„) must be 
chosen in QS ( — 1) and must satisfy lim„_ i . 0O n^ n — 1. The most simple example of stepsize belonging to 
QS (—1) and such that lim„_ ! . 00 nj n = 1 is (7„) = (n -1 ). For this choice of stepsize, the estimator /„ 
defined by (HJ equals the recursive kernel estimator introduced by Wolverton and Wagner (1969). 

To establish pointwise LDP for /„ in this case, we need the following assumptions. 

(LI) K : M. d — > R is a bounded and integrable function satisfying f Rd K (z) dz = 1, and limi^n^oo K (z) = 
0. 

(L2) i) (h n ) = {cn- a ) with a G ]0, l/d[ and c> 0. 

a) hn) = {n^ 1 )- 

The following Theorem gives the pointwise LDP for /„ in this case. 
Theorem 1 (Pointwise LDP for Wolverton and Wagner estimator) . 

Let Assumptions (LI) and (L2) hold and assume that / is continuous at x. Then, the sequence 
(fn (x) — f (x)) satisfies a LDP with speed {nhf^ and rate function defined by @. 

2.1.2 Choices of (7„) minimizing the variance of /„ 

It was shown in Mokkadem et al. (2009) that to minimize the asymptotic variance of /„, the stepsize (7„) 
must be chosen in QS (—1) and must satisfy limn._j.oo "7n = 1 — ad. The most simple example of stepsize 
belonging to QS (—1) and such that lim„_ >00 nj n = 1 — ad is (j n ) = ((1 — ad) n ), an other stepsize 

satisfying this conditions is (7™) = (h^ (J_l-=i hf) 1 ) • F° r this last choice of stepsize, the estimator /„ 
defined by ([1} produces the estimator considered by Deheuvels (1973) and Duflo (1997). 
To establish pointwise LDP for /„ in this case, we assume that. 
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(L3) i) (h n ) G GS (-a) with a G ]0, l/d[. 
H) iln) = (K (ELi^fe)" 1 )- 

The following Theorem gives the pointwise LDP for f n in this case. 
Theorem 2 (Pointwise LDP for Deheuvels estimator). 

Let Assumptions (LI) and (L3) hold and assume that / is continuous at x. Then, the sequence 
(fn (x) — f (x)) satisfies a LDP with speed (nh^) and rate function defined by ([3]). 



2.2 Pointwise MDP for the density estimator denned by the stochastic ap- 
proximation algorithm ([1]) 

Let (v n ) be a positive sequence; we assume that 

(Mf) K : M. d — > R is a continuous, bounded function satisfying J Rd K(z)dz = 1, and, for all j 6 
{1, . . . d}, J R zjK (z) dzj — and J Rd z?\K (z) \dz < oo. 

(M2) i) ( 7n ) G GS (-a) with a G ]l/2, 1]. 
m) (/i n ) G £7<S (—a) with a G ]0, a/d[. 
Hi) lim n ^oo (nj n ) g] min{2a, (a — ad)/2}, oo]. 

(M3) / is bounded, twice differentiable, and, for all i,j G {1, . . .d}, d 2 f / dxidxj is bounded. 

(M4) lim^oo v n = oo and lim^oo jnvl/h^ = 0. 

The following Theorem gives the pointwise MDP for f n . 

Theorem 3 (Pointwise MDP for the recursive estimators defined by Q). 

Let Assumptions (Ml) — (MA) hold and assume that / is continuous at x. Then, the sequence 
(fn (x) — / (x)) satisfies a MDP with speed (h^/ (lnV 2 )) and rate function J a ,u,x defined in . 



2.3 Uniform LDP and MDP for the density estimator defined by the stochas- 
tic approximation algorithm ([1]) 

To establish uniform large deviations principles for the density estimator defined by the stochastic ap- 
proximation algorithm (TT|) on a bounded set, we need the following assumptions: 

(Ul) i) For all j G {1, . . . d}, J R ZjK (z) dzj = and J Rd Zj\K (z) \dz < oo. 
ii) K is Holder continuous. 

(U2) / is bounded, twice differentiable, and, sup xeH d \\D 2 f (x) \\ < oo. 



(U3) lim. 



7„< log(l/fr„) 

71— J-OO fad 



and lim r - 



7„7J ra log Vn 



0. 



Set U C K d ; in order to state in a compact form the uniform large and moderate deviations principles 
for the density estimator defined by the stochastic approximation algorithm (fTJ) on U, we set: 



\U,oo-l a 



In 1 



9u (S) 
9u (S) 



f\\v,c 

s 



1 i 1 + lmfc) 



\\I\\U,oo 

8 2 (2-(u-ad)£) 
211/11^00/^ K2(z)dz 

mm{gu (6),g v (-5)} 



when v n = 1 
when v n = 1 
when v n — > 00 



(LI) and (L2) hold 
(LI) and (L3) hold 
(Ml) - {MA) hold 



where ||/||u,cx> = snp xEU \f (x)\. 

Remark 1. The functions gu (■) and 9u (■) are non-negative, continuous, increasing on ]0,+oo[ and 
decreasing on ]— 00, 0[, with a unique global minimum in (gu (0) = gu (0) = 0). They are thus good 
rate functions (and gu(-) is strictly convex). 

Theorem H] below states uniform LDP on U in the case U is bounded, and Theorem [S] in the case U 
is unbounded. 
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Theorem 4 (Uniform deviations on a bounded set for the recursive estimator defined by ([T])). Let (Ul) — 
(U3) hold. Then for any bounded subset U of K rf and for all S > 0, 



lira ^logl 



SUpV n \f n (x) - f(x)\>6 

x£U 



-9u («*) 



(7) 



To establish uniform large deviations principles for the density estimator defined by the stochastic 
approximation algorithm (|TJ) on an unbounded set, we need the following additionnal assumptions: 

(U4) i) There exists (3 > such that J Rd ||a;|| /3 / (x) dx < oo. 
ii) f is uniformly continuous. 

(U5) There exists r > such that z ||z|| r iir (z) is a bounded function. 

(U6) i) There exists £ > such that J Rd ||z||'> \K (z)\ dz < oo 

ii) There exists 77 > such that z 1— > (z) is a bounded function. 

Theorem 5 (Uniform deviations on an unbounded set for the recursive estimator defined by (JlJ). Let 
(Ul) - (U6) hold. Then for any subset U of R d and for all S > 0, 



-~ gu (6) < hminf^logl 

n 

< hmsup^jMogP 



SUpV n \f n (x) - f (x) | >S 

supu n |/„ (a;) - / (x)\ >S 



< -■ 



/3 + d 



9u (6) 



The following corollary is a straightforward consequence of Theorem [5] 

Corollary 1. Under the assumptions of Theorem^ if J Rd ||a;||^/ (x) dx < 00 for all £ in K, then for any 
subset U of 



hm ^logl 

n->oo ft" 



supu„ |/„ (x) - f(x)\>6 

x£U 



-9u (S) 



(8) 



Comment. Since the sequence (sup^gj/ \f n (x) — f (x)\) is positive and since gu is continuous on 
[0, +oo[, increasing and goes to infinity as 5 — > oo, the application of Lemma 5 in Worms (2001) al- 
lows to deduce from ([7]) or that sup^g^ \f n (x) — f (x)\ satisfies a LDP with speed (7,7 1 /i^) and good 
rate function gu on K + . 



3 Proofs 

Throught this section we use the following notation: 



n n = H(i- 7j ), 

3=1 



(9) 



Throughout the proofs, we repeatedly apply Lemma 2 in Mokkadem et al. (2009). For the convenience 
of the reader, we state it now. 

Lemma 1. Let (v n ) 6 QS (v*), ("f n ) £ QS (—a), and m > such that m — v*£ > where £ is defined 
in (gj). We have 



f 



hm a:Vn l -^ = 

fe=l s 



n— >+oo 



Moreover, for all positive sequence (a n ) such that lim„_j. +00 a„ = 0, and for all 5 G 

r— m Tfc 



lim 7>„n™ 

n— J-+oo 



J2ll^^a k +6 



0. 
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Noting that, in view of |T|), we have 
f n (x)-f(x) = (l-7n)(/«-i(z)-/(a:))+7»(MaO-/(aO) 



E 



fe=i 



n a-7i) 



nc 1 -^) 



(/o(a0 -/(*)) 



7fc (Z fe (x) - / (x)) + 7„ (Z„ (x) - / (a;)) + 

= n„ n^Sfe (z k (x) - f (x)) + n„ (/ (x) - / (x)) . 

fc=l 

It follows that 

n 

e [/„ (a)] - / (*) - n„ £ n-^fc (e [z h (x)] - / (x)) + n„ (/ (x) - / (x)) . 

k=l 

Then, we can write that 

n 

fn (x) — E [/„ (x)] = U n J2^k 1 7k(Z k (x)-E[Z k (x)}) 

k=l 

n 

= U n ^2u^ 7k K d (Y k -E[Y k }) 
fc=i 

Let (vPn) and (B n ) be the sequences defined as 

n 

* n (x) = n„^n^ 7 ^fc d (n-E[Y fc ]) 
fe=i 

5„(x) = E[f n (x)]-f(x) 

We have: 

/„ (x) -f(x) = (x) + B n (x) (10) 

Theorems [1] [21 El HI and [S] are consequences of (jTUJ) and the following propositions. 
Proposition 1 (Pointwise LDP and MDP for (f„)). 

1. Under the assumptions (LI) and (L2), the sequence (/„ (x) — E (/„ (x))) satisfies a LDP with speed 
(nhfj and rate function I a ,x- 

2. Under the assumptions (LI) and (L3), the sequence (f n (x) — E (/„ (x))) satisfies a LDP with speed 
[nhfA and rate function I x . 

3. Under the assumptions (Ml)— (Mi), the sequence (v n ^ n (x)) satisfies a LDP with speed (h^/ YYnVn)) 
and rate function J a , a ,x- 

Proposition 2 (Uniform LDP and MDP for (*„)). 

1. Let ({71) - ({73) hold. Then for any bounded subset U of R d and for all 6 > 0, 

,,2 



l im ^io g ] 



sup v„ |* n (x)| > <5 



-5c/ (6) 



2. Let (C/l) - (t/6) hold. Then for any subset {7 of M'* and for all S > 0, 



-&/(<*) < liminf^Uog] 

2 

< lim sup log P 



supu n (x)\>5 

SUpV n |*„ (x)| > 6 

x<£U 



< 



9u (8) 



The proof of the following proposition is given in Mokkadem et al. (2009). 
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Proposition 3 (Pointwise and uniform convergence rate of (£?„)). 
Let Assumptions (Ml) - (M3) hold. 

1. If for all i, j G {1, . . . d}, d 2 f / dxidxj is continuous at x. We have 



If a< a/(d + A), then 



If a > a/(d + 4), then 



2. If (t/2) holds, then: 
If a < a/(cf+4), then 



If a > a/(d + 4), then 



B n (x) = O (h 2 n ) 



B n (x) = o y\ppjh^ 



sup \B n (x)\=0(h 2 n ) 



SUp \B n (x)\ = O (Jjnhn' 

xeR d \ 

Set x G K d ; since the assumptions of Theorems [T] and [3] guarantee that lim n _).oo B n (x) = 0, TheoremQ] 
(respectively Theorem [5]) is a straightforward consequence of the application of Part 1 (respectively of 
Part 2) of Proposition [TJ Moreover, under the assumptions of Theorem [3J we have by application of 
Propostion[3J limn-j.oo v n B n (x) = 0; Theorem[3]thus straightfully follows from the application of Part 3 of 
Proposition [T] Finaly, Theorem [J] and [5] follows from Proposition [2] and the second part of Proposition [3] 
We now state a preliminary lemma, which will be used in the proof of Proposition [1] 
For any Set 



Ki,x (u) 

A^ 2 (u) 
Af («) 



loo- £ 



cxp u 



K 

f (x) (lp a (u) - u) 
f(x) (ip (u) -u), 
u 2 

2 (2 - (a - ad) £) ' 



/(x) / X 2 (*)(fe 



Lemma 2. [Convergence of A„ )X ] 

1. (Pointwise convergence) 

If / is continuous at x, then for all u G 



lim A„ tX (u) = A x (u) 



(11) 



where 



A x {u) 



A^' 1 (u) when v n = 1 
A£' 2 (it) when w n = 1 
A^ (u) when w„ — > oo 



(LI) and (£2) hold 
(LI) and (L3) hold 
(Ml) - (A/4) hold 



2. (Uniform convergence) 

If / is uniformly continuous, then the convergence (fTTj) holds uniformly in x G U. 

Our proofs are now organized as follows: Lemma|2]is proved in Section l3~Tl Proposition[T]in Section l3~4l 
and Proposition [5] in Section [ 
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3.1 Proof of Lemma [21 

Set u £ R, u n — u/v n and a n = h^j^ 1 . We have: 

v 2 

k n ,x (u) = — log E [exp (u n a n ty n (x))] 



^logE 



^£logE 



exp u n a n U n Hfc 1 (n - E [F fc ]) 

V fe=i 
a„n. 



fe=l 



exp u n — — Yi 



By Taylor expansion, there exists c ktTl between 1 and 



logE 



exp u 



-n 



' a k U k 

and A n ^ x can be rewriten as 

9 n 

r 



E 



exp u n — —Yk I - 1 



ClkJlk 



exp ( u r 
1 



fc=l 
a fc n fc ^ 



ZC fc,n 



E 



such that 

a„n r 



exp | u„ — — —Yk ) — 1 
a k U k 



2 



fc=i 



exp u n —^-Y k I - 1 



Ofcllfc 

nnE n fe la fc lE M 



2 11 i 

V — 

za " fe =i 



E 



Q. n II n 

exp i u„ — I - 1 



(12) 



fc=i 



First case: v n oo. A Taylor's expansion implies the existence of c' k n between and u n "^n^ Y k such 



that 

E 



exp u n — — Y k I - 1 



2 V a fc n fc I 



6 V a k U k J 



Y*e^ 



Therefore, 



A„, x (u) 



fe=i 



Za » fe=l C fc,« 



E 



exp u. 



6 

' a k U k 



Y k -1 



fc=i 

2 



with 



1 - f 

= -/(x)u 2 aii n 2 ^n- 2 a-S / K 2 (z)dz + ii« («) 

2 fc=l ^ 

i U 2 a„n 2 ^ Ufa^ lk / i^ 2 (z) [/ (x - zh k ) - f (x)} dz 

2 k=l J ^ d 



1 lt J 

6 v n 



x 2 n 3 ^n 



k 3 ^ 



Y*e<- 



2 '* 



fe=i 



2a„ 



fe=l "*v 



E 



(13) 



exp [ u„ — ^~Y k I - 1 



Since / is continuous, we have lim/^oo \f (x — zhk) ~ f — 0, and thus, by the dominated convergence 
theorem, (Ml) implies that 



lim / K 2 {z)\f{x-zh k )-f{x)\dz = Q. 
Since (a„) G QS (a — ad), and lim rl _ i . 0O (rvyn) > (a — ad) /2. Lemma [T] then ensures that 



i 



fe=i 



(2 - (a - ad) 



(14) 
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it follows that lim,- 



R { nl (U) 



0. 



Moreover, in view of ©, we have \Y k \ < \\K\\ , then 



Cfe, 



< 
< 



«n| 11*11, 



5 1. 



(15) 



Noting that E\Y k f < h d k 



constant c\ such that, for n large enough, 



J Rd \K 3 (z) \ dz. Hence, using Lemma [T] and (|T5|) . there exists a positive 



fe=i 



Y?e< 



< c\e 



"nlllA'H 



ll/IL / \K 3 (z)\dz 



(16) 



which goes to as n — > oo since v n — > oo. 
Moreover, Lemma [1] ensures that 



Za " fe=i C fe,« 

2 



OnIl ra 

CXp U n =— JTfc I - 1 



< 



E 

2a ^ V 



fe=i 



cxp u 



1 a k Tl k 



Y k - 1 



< 



2 " 

fc=i \ fc=i / 

= o(l) (17) 

The combination of (fT!)]) and pT|) ensures that limn^oo i?n 2 ^ (it) = 0. Then, we obtain from (fT5)) and 
(HI, lim An tX (u) = Af (u). 



Second case: (w n ) = 1. It follows from (fT2|) that 



An,* (it) 



n 

-Ve 



fc=l 



CXp It — Yfc - 1 

flfcllfc 



J_ A 

fe=l 



2a « C fc,n 



E 



cxp I u ^n k Yk I - 1 



-wn„E n fe la fe lE [ y fc] 



k=l 



l n k=1 JM." 



f{x)dz-ua n Y j ni l lk [ K{z)f{x)dz 
k =i J ^ d 



i - r r 

f( x ) — Y, h t / («p(uK, fc iir(«))-l)-iiV nifc A'(z) 

a n k=l URd 



dz 



(18) 



with 



#S («) 

(«) 



1 ^ 1 

2^ E 

/a ™ fe =i C M 



6XP ' aOI/t ' ~ 



a n k=1 jRf 

n „ 

-ull n Y Tl- 1 ^ / A' (z) [f (x - zh k ) - / (x)] dz. 
k=i jRd 



[f (x - zhk) - f (z)] dz 
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It follows from (fTT|) , that lim, woo Fl£} x (it) 
Since |e* — 1| < |i| e'*' , we have 



= 0. 



u ^k K{z) 



[f (x - zh k ) - f (x)] 



< -f>2 I 

a n . 1 JR' 

n „ 

+ | u |n„^n- 1 7fc / \K(z)\\f( x -zh k )-f(x)\dz 
k =i jRd 

< HeWll^ll^n„Vn^7 fc / \K(z)\\f(x-zh k )-f(x)\dz 

k=l J ^ d 

n ,. 

+ \u\u n Vn^S / |^WII/(x-z/i fe ) -/(x)|dz 



< 



fc=i 

H( e HI«»+l)n„y;n^7fc / \K(z)\\f{x-zh k )-f(x)\dz 



In view of Lemma [T] the sequence (ll„ n fe 1 7fc) is bounded, then, the dominated convergence the- 

orem ensures that lim n _ i . 00 R^h} x (u) = 0. 

In the case / is uniformly continuous, set e > and let M > such that 2 H/H^ fi\ z » <M \K (z)\ dz < e/2. 
We need to prove that for n sufficiently large 



sup 



\K (z)\\f (x - zh k ) - f (x)\dz < s/2 



x£R d J\\z\\<M 

which is a straightforward consequence of the uniform continuity of /. 
Then, it follows from flU}, that 



lim^W - lim / (x) ^ V hi / [(exp(uV n , Jt ii:(z))-l)-«V n , fc A'(z)]d2 (19) 

n->oc n^oo fog f— J J R d 



In the case when (u„) = 1, (LI) and (L2) hold 
We have 



n fc 



n d-v) 

j=k+l 
k 



then, 



V, 



n.k ■ 



afellfe 
k 
n 



ad 



Consequently, it follows from (fTU|) and from some analysis considerations that 



lim A n , x (it) = f (x) 

n— too 



-ad 



(exp (us ad K (z)) - 1 - us ad K (z)) ds 



dz 
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In the case when (v n ) = 1, (LI) and (L3) hold 
We have 



Ilk 




n V 3 " 1 h d 
j=k+i 2^i=i <H 

K KEUhj 
ikK' 



v n , k = 1. 



then, 



Consequently, it follows from (fT§|) that 

lim A nx (u) = f (x) [(exp(uK (z)) — 1) — uK (z)]dz 

n->oo J Kd 

and thus Lemma 1 is proved. 

3.2 Proof of Proposition Q] 

To prove Proposition [TJ we apply Proposition 1 in Mokkadem et al. (2006), Lemma [2] and the following 
result (see Puhalskii, 1994). 

Lemma 3. Let (Z n ) be a sequence of real random variables, (yn) & positive sequence satisfying lim^—j.oQ v n — 
+oo, and suppose that there exists some convex non-negative function T defined on R such that 

Vu e R, lim — logE [cxp {uv n Z n )\ = T (u) . 

If the Legendre function T* of T is a strictly convex function, then the sequence (Z n ) satisfies a LDP of 
speed (v n ) and good rate fonction P*. 

In our framework, when v n = 1 and j n = we take Z n = f n (x) — E(/„ (x)), f n = n/i^ with 
/i„ = cn~ a where a € ]0, l/d[ and T = A^' 1 . In this case, the Legendre transform of T = A^' 1 is the 

rate function I a x : t — > f (x) I a ^jj^y + lj which is strictly convex by Proposition 1 in Mokkadem et al. 

(2006). Farther, when v n = 1 and j n = h* (J2k=i h t) > we take %n = fn i x ) - E (f n v n = nh* 
with h n £ £/<S (—a) where a e ]0, l/d[ and T = A^" 2 . In this case, the Legendre transform of T = A x ' 2 

is the rate function I x : t — > f (x) I (^jf^ + which is strictly convex by Proposition 1 in Mokkadem 

et al. (2005). Otherwise, when, v n — > oo, we take Z n = v n (f n (x) — E (f n (x))), v n = h^/ (jnVn) and 
r = A^; r* is then the quadratic rate function J a , a .x defined in ([5]) and thus Proposition Q] follows. 

3.3 Proof of Proposition [2] 

In order to prove Proposition [2J we first establish some lemmas. 
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Lemma 4. Let <f> : R + — ► K be the function defined for S > as 



«) [1 + jj^—j when u„ = 1 , (LI) and (L2) hold 
= <| (tp'y 1 (l + iijii^ j when « n = l , (LI) and (L3) hold 
Ofeg^fa' 00 when . (Ml) - (MA) hold 

1. sup MgR {uS — sup xGU A x (u)} equals gu (5) and is achieved for u — 4> (5) > 0. 

2. sup MgR {— uS — sup^gjy A x (u)} equals gu (S) and is achieved for u = <f> (—5) < 0. 

Proof of Lemma [4] . We just prove the first part, the proof of the second part one being similar. 

• First case : v„ = 1, (LI) and (L2) hold. 

Since e* > 1 + 1, for all t, we have ip a (u) > u and therefore, 

u6 - sup A x (u) = uS - WfWu.oc (ip a (u) - u) 

x£U 



ll/l 



U,oo 



The function u n> uS — sup^g^ A x (u) has second derivative — ||/||[/,ooV'o ( u ) < anc ^ thus it has a 
unique maximum achieved for 



"^ wr '( 1 + iLfib) 



Now, since i/j' a is increasing and since ip' a (0) = 1, we deduce that uq > 0. 

Second case : v n = 1, (LI) and (L3) hold. 

Since e* > 1 + 1, for all t, we have ip (u) > u and therefore, 

uS - sup A x (u) = uS - WfWu^iip (u) ~ u) 
xeu 



l/lk. 



The function u n> uS — sup^g^ A x (u) has second derivative — 1|/|| u,cxi>" ( u ) < and thus it has a 
unique maximum achieved for 



Now, since ip' is increasing and since ip' (0) = 1, we deduce that uq > 0. 
• Third case v n oo and (A/2) holds. In this case, we have 



uS - sup A x (u) = uS - — — — — ll/llu-.oo / K (z)dz. 

xeu 2(2-(a-arf)4) J Rd 

In view of the assumption (M2), we have > (a — ad) /2, then the function u H> u<5 — 
supj.gy A K (it) has second derivative — ^ 2 -(a-ad)(,) \\f\\u,oo J M d Kr (z) dz < and thus it has a unique 
maximum achieved for 

= 5(2-(a-ad)Q 
° \\f\\u,ooJ Md K2(z)dz 



Lemma 5. 



In the case when (v n ) = 1 and (7„) = (n x ), let (LI) and (L2) hold; 

In the case when (v n ) = 1 and (7„) = (h c ^ (J2k=i hf) > ^ (-^-0 an d (^3) hold; 
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In the case when v„ -> oo, let (Ml) - (M4) hold. 
Then for any S > 0, 



2 

lim ^ log sup P [« n *„ (x) > 5} = - 9u (5) 



2 

lim ^g" log sup P [v n y n (x) < -5} 
logsupP[v„ |* n (x) < -5} 

xeU 



lim 

n— >qg h 



-9u (S) 

-go (S) 



Proof of Lemma [5} The proof of Lemma [3] is similar to the proof of Lemma 4 in Mokkadem et al. 
(2006). 

Lemma 6. Let Assumptions (Ul) — (U3) hold and assume that cither (v n ) = 1 or (U4) holds. 



1. If U is a bounded set, then for any 8 > 0, we have 



lim ^MogP 



SUpV n |*„ (x)| 
x£U 



< -gu (S) 



2. If U is an unbounded set, then, for any b > and S > 0, 

„2 



lim sup log P 

hi 



n->oo '' n 



SUp Un|*n(s)| 
xeW,||a:||<«; n 



<db-gu (6) 



where iu„ = cxp f^VJ • 



Proof of Lemma [6j Set p G ]0,<5[, let /3 denote the Holder order of K, and \\K \\h its corresponding 
Holder norm. Set w n = exp {^^j and 



2\\K\\ H v n Ii n Y: k=1 ^l 1 lkh 



A' 



We begin with the proof of the second part of LemmalHl There exist AT' (n) points of P rf , 2/1,2/2, • • • j yjv'^n) 
such that the ball {x G M d ; ||x|| < w n } can covered by the N' (n) balls B 4 (n) = |x G M. d ; \\x — y^ n) || < 



and such that N' (n) < 2 \^r l J ■ Considering only the N (ri) balls that intersect {x E U; ||x|| < w n }, 
we can write 

{xGU;\\x\\<w n }cU^Bl n) . 
For each i G {1, . . . , N (n)}, set x| n) G B 4 (n) n f7. We then have: 

N(n) 



sup i; n |* n (x)| > S 

x£U,\\x\\<w n 



< E 

i=l 



sup u„ |*„ (x)| > 5 



x£B) 



< N (n) max ] 

l<i<N(n) 



sup U„ |* n (x)| > 5 



x£B) 
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Now, for any i £ {1, . . . , TV (n)} and any x 6 B\' 

n 

+t» n IIn ^ U^Jkh'f. 

fc=l 

71 



K 



fc=i 



A" 



x - X k 
h k 



fx - X k 



-K 



K 



h k ) 



< V r , 

< V r , 

< V r , 



*„ (^ n) ) | + 2v n \\K\\ H U n J2 l lk h^ d 

k=l 
n 

*„ (4 n) ) | + 2,„|| A|| ff n„ 2 il^^^ 



lk-4 n) ll 



fc=l 



h k 



Hence, we deduce that 



sup v n |*„ (x)| > S 

x£U,\\x\\<w n 



< TV (n) max ] 

l<i<N(n) 



< TV (n) sup 1 



>5-p 



>S-p 



Further, by definition of TV (n) and w n , we have 



log TV (n) < log TV' (n) < db—^r + (d + 1) log 2 - dlog R r , 



and 



^■logRr,-^ 



hi 



Moreover, we have ( h [ n +fi) ) 6 GS {-a (d + /?)). LemmaQ] ensures that 



log y o-log(2|| J PC|| H )-log« n -log (n n ^2u^\ k h 



k 



fe=l 



n 



fe=l 



then, in view of (US), we have 



2 

li m sup 2ga l g iv(n) < d6 



The application of Lemma [S] then yiels 



lim sup log 1 



SUP V n |*„ (x)| > (5 
a;ezt/,||:c||<'u; T1 



(20) 



< lim sup 2ga log TV (n) - gu (6 - p) 

n—¥oo fir, 



< db-gu(S- p). 

Since the inequality holds for any p £ ]0,<5[, part 2 of Lemma[B]thus follows from the continuity of gu- 

Let us now consider part 1 of Lemma [5] This part is proved by following the same steps as for part 
2, except that the number TV (n) of balls covering U is at most the integer part of (A/R n ) d , where A 
denotes the diameter of U. Relation ((2T)|) then becomes 

2 

lim sup log R n < 

71— >00 



and Lemma [6] is proved. 
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Lemma 7. Let (£71) i), (M2) and (U6)i) hold. Assume that either (v n ) e 1 or {US) and (U6)ii) hold. 
Moreover assume that / is continuous. For any b > if we set w n = exp ^ ^ h ™ 2 ^ then, for any p > 0, we 
have, for n large enough, 



sup v n n n ^n k 1 '-f k h k d 



Proof of Lemma We have 



fe=i 



K 



fe=i 



3 



A" 



X - Az 



< 



n . 

= v n u n V iliS* / # 



(z) / (x — zhk) dz. 



First, Lemma [I] ensures that 



nn^n^Sfc = i + o(i). 



(21) 



(22) 



fe=i 



Set p > 0. In the case (u„) ee 1, we set M such that |/oo|| Jj| z ||>j\/ \K ( z )\ < p/2; it follows from l(2"2"j) 
that 



fe=i 



3 



A 



x - A fe 



<f + f{x) 



\K{z)\dz 



\z\\<M 



fc=l ->ni>m 



Lemma [7] then follows from the fact that / fulfills (U6)ii). As matter of fact, this conditions implies 
that lim... ii^^ x£ jj f (x) = and that the third term in the right-hand-side of the previous inequality 
goes to as n — > oo (by the dominated convergence). 

Let us now assume that liirin^oo v n = oo; relation (|2ip can be rewritten as 



fc=l 



K 



x-X k 



n ,. 

= ^iL^iLr^ / 

fc=l ^11 
n « 

r~; J\\ 



z\\<w„/2 



K (z) / (x — zhk) dz 



\\z\\>w n /2 



K (z) f (x — zhk) dz. 



First, since ||x|| > w„ and ||z|| < w n /2, we have 

\\x - zh k \\ > w n (l-hi/2) 

> w n /2 for n large enough. 

Moreover, in view of assumptions (C/3), for all £ > 0, 



lim — j — lim exp < — £6 

n— >oo lt S n— yoo 



InVl 



1 



K log Vn 
&ht 



Set Mf = swp xGRd ||x|| 7 '/ (x). Assumption (U6) ii) and equations 
large, 



0. (23) 
implie that, for n sufficiently 



sup v Il L T „^L T fc 1 7 fc / 

\\x\\>w n fc=1 J\\ 

n ., 

< M f sup u„n„V'n^ 1 7 fc / 

ii*ii>»» k=i 

<2 r, M f ^ f f \K(z)\dz 



<\\z\\<w n /2 



K (z) f (x — zhk) \ dz 



\z\\<w n /2 



\K(z)\\\x-zh k \\- r 'dz 



- 2 



15 



Moreover, in view of (t/3), (U6) i) and (j2"2")h (j2"3")h for n sufficiently large, 



n ,. 

sup «„n n n^ 1 7fc / 

x||>iu„ , , J\\ 



\K (z) f (x- zh k )\ dz 



< 2 C M 



fc=i 
v 



Wn J\\z\\>w n /2 



||*||>«;„/2 

z|| c \K{z)\dz 



This concludes the proof of Lemma [7] Since if is a bounded function that vanishes at infinity, we have 
limii^n^oo \ty n (x)| = for every n > 1. Moreover, since K is assumed to be continuous, ty n is continuous, 
and this ensures the existence of a random variable s n such that 

|*„ (s„)| = sup |*„ (x)| • 

x£U 

Lemma 8. 

Let Assumptions (Ul) — (t/3), (U4)ii) and (f/5) hold. Suppose either (v n ) = 1 or (H6) hold. For any 
b > 0, set w n = cxp (b j ; for any (5 > 0, we have 



limsup^jMogP[||s n || >u) n and |*„(s„)| > <5] < -6/3 



(24) 



Proof of Lemma [8j We first note that s„ € ?7 and therefore 
> w„ and v„ |* n (s n )| > 8 



=> || || > w n and u n 



iini:n^W^( £! ^) 



fe=i 



| Sn || > w„ and v„II n £ 1 7 fc /i fc d 



fc=i 



A" 



S» — Afe 



sup _u„n„^n^ 1 7 fe/ifc d E 

||K||>«; n ,a:Gi7 



fe=l 



A 



hk 



> ,5. 



Set p G ]0, 5[; the application of Lemma [7] ensures that, for n large enough, 

||s n || > w n and v n |*„ (s„)| > (5 

n / y 



=>■ II Sn II > w n and w,- 



fc=i 



hi 



>S-p. 



Set k = sup^ggd ||x|| 7 |A(x)| (see Assumption (f 5)). We obtain, for n sufficiently large, 
||s„|| > w n and v n (s„)| > (5 



> iu„ and 3fc G {1, . . . , n} such that 



A 



s„ - A* 



/'A 



> 5-/3 



> W r , 



hi 



and 3fcG{l,...,n} such that Kh\ > — \\s n - A fc || 7 (6 - p) 



>w n and 3k G {1, . . . , n} such that |||s„|j - ||A" fe ||| < 



> w n and 3k G {!,... ,n} such that \\Xk\\ < \\s r , 



KVnhl d 

5-P 



VnK d 



5-P 



> w n and 3k G {1, . . . , n} such that ||-X"fe|| < w n (1 — with 



U n ,k = W n Vn\h k 



S-p 
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Moreover, we can write u n ^ as 

/ hi 
u«i = exp -b 



InVl 



1 "fnV^logVn 7 - d^ n v^\og(hk) 



try hi 



fy) 



hi 



5-p 



and assumption (US) ensure that linin^oo u n _ k = 0, it then follows that 1 — u n _ k > for n sufficiently 
large; therefore we can deduce that (see Assumption (U4)i)): 

n 

P [|| s ri || > Wn an( l V n \ 1 ^ n (s„ )\>s\ < Y, p [w x k\\ f3 

>W%(1- U n ,k) 

1=1 

n 

< "£E(\\x k f)w-e(i- Un , k )-e 

1=1 

< nE(||Xif)«;-^ max (l-w„, fc ) _/3 . 



Kfe<n 



Consequently, 



logP[||s n || > w n and |*„ (s n )\ > 5} 



< 



hi 



logn + logEdlXxf) -6/3 



/31og max (1 - u n , k ) 



and, thanks to assumptions (C/3), it follows that 



2 

limsup^logP[||s„|| >w n and v n \V n (s n )\ > S] < -bp, 



which concludes the proof of Lemma HI 



3.4 Proof of Proposition 

Let us at first note that the lower bound 

2 

hminf ^Mogl 



SUpV n (x)\>6 



> -9u (S) 



(25) 



follows from the application of Proposition [T] at a point xq £ U such that / (xo) = H/Hc^oo- 

In the case U is bounded, Proposition [2] is thus a straightforward consequence of (|23|) and the first part 

of Lemma IB1 Let us now consider the case U is unbounded. 

(h d \ 
& 7 ^ 2 ) ■ Since, by definition of s n , 



supi; n \^ n (x)\ > S 

x<£U 



< 



sup v n |<3> n (x)\ > 5 
xeu,\\x\\<w„ 



it follows from Lemmas [5] and [5] that 



2 

hmsup-^logP 



sup (a;) | > 5 



and consequently 



r 7«< , t 

hm sup log 1 

n— Kx> 



supw„ (a;) > (5 
xeu 



' [\\s n \\ > w n and v n \ (x)\ > 6} , 



< ma,x{— b(3;db — gu (5)} 



< inf max {— 6/3; <i6 — gu (S)\ . 

b>0 



Since the infimum in the right-hand-side of the previous bound is achieved for b = gjj (6) / (/3 + 6) and 
equals ~Pgu I (P + d), we obtain the upper bound 



2 

hm sup log 

71— KX> ftn 



supv„ |*„ (a;) I > <5 



< 



P + d 



~9u (6) 



which concludes the proof of Proposition [2 
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